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For years the Bureau of School Prcgrais Evaluation of 
the New York State Education Departient has studied relationships 
between school processes and stjtdent achieveient. The pri■a^y 
strategy has beer (1) If) us€ luitiple regression analysis to obtain 
relationships, (2) to use school/district average &<?ores as the data 
for analysis, and (3) to eliii'nate, to the extent ^ossiblei, 
differences between districts that lay be attributed to socioeconomic 
conditions and that are not controllable by the schools. This xpaper 
briefly describes a series of analyses carried out to study 
relationships between a nuiber of school processes and student 
achievement on third- and sixth-grade latheiatics and reading tests. 
Process variables fell under the general headings of teacher, ' 
financial, and student. Several teacher variables were found to 
relate to achievement, but no consistent pattern of relationships was 
found between expenditure variables and achievement. The positive 
results for several teacher variables support logic and conventional 
wisdom. The higher the salary and the higher the percent of certified 
teachers and teachers with graduate, credit, the higher district 
achievement is Jikely to be. These analyses^ do not reveal whether 
cause and effect are inherent iji these relationships, (Author/IRT) 
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STUDYING* SCHOOL PROCESSES THROUGH 
THE ANALYSIS Of SCHOOL DISTRICT DATA 

David J. Irvine* * 



Tl?e Bureau of School Programs Evaluation of the New YorkTtate 
Education Department has for a number of years^ studied telationships- 
, between schor- procejssea and' achievement. The primary strategy has 

been: (1) to use multiple regression analysis to obtain relationships; 

(2) to use school district average scores as th^ data for analysis; and 

(3) to eliminat;e,^ to the extent possible, differences between districts 
which may be attribijted to socioeconomic conditions and o^j^r variables 

r not controllable by the schools. ThuS' far, the criteria have been 

, •. ' • • • • , ' ■ ' 

' restricted to reading and matheroatids test scores. 

^ This paper brief ly. describes a series of analyses carried out to 

srudy relationships, \^etween a number of school processes and achievement. 

Mort:^ detailed and technical inf ofni^ation regarding specific analyses can 
^be found In papers produced by the bureau. ..They are listed under 

References and are available on r^quest^' The ''prefiertf paper does not 

• -* • * / ' 

attempt to describe the technical dspects^of individual studies. 

For purpoies- of tjris paper; the term' school processeg' ^e used to 

• include thpse factors under the co^|rol ^of the school or school district. 



*I wish to- acknowledge the efforts of my colleagues in the Bureau/of 
School Prograc^s Evaluation who carried out the^analyses described here 
and assisted, in the preparation of this paper, they are Gerald H. 
Wohlferd, Guy D. Spath, Philip J. Pillswofth, and Gennaxto DiGiovanni.r- 



For example, expend itureV are defined as processes even though they ^ 
do not iiT.pinge directly on students, ^ * 

Data Sources 

Ihi New York State Education Department has ' an unu^lially la^rge 

and varn_«d ooc 1 of educational data which is updated each year t^hrough 

the Inf orTa t^log ^C'^nter on Educatiot?, Data of several types ar^ * 

collected frorr each school district*. The data files include;"^ 

1. Ihe Basic Educational Data System (Bffig), 
* " which contains data on each district, school, 

and prof essiona-l staff member in the state, - 

The Pupil Evaluation Program (PEP), which each 
* year tests virtually all ^tudents in grades 

three, six, and ni/ne on tWadinc and mathegiarfTci, 
Retarded students and non- EngliHii speaking 



students are exempt from taking the test^, 
Results are avaflabie for eac^yschool Arid each 
^ ^ district, ' 

3, F?^anclal reports, which show income and 
expenditures. of school districts, 

» 4, U,S.* Census data, whith have been coHected 
or estimated for each school district in the 
state, ^ 

5, School census and enrollment, which contains 
data for schools and districts. 



Criterion Variables 

The achievement criteria used to assess the importance of school 

processes were district mean scores on the PEP tests: ^ 

Third-grade Reading 
Third-grade Mathematics 
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^ Sixth-^grade Reading 

Sixth-grade Mathei^atics 

Data for the 1971-72 and, in most cases, 1972-73 school years were 

examined, * * * 



^k)ncontrollable Variables 

Several iocioeconomic variables and other variables not con- 

\ 

' trollabie by the schools were used to control statistically for the 

effects of nonschool factors: - ' 0 

Total Population of the District * 
' Percent Ruiral Population ^ 

Percent of Children Living in a Father/Mother Family 
\ Percent of Owner Occupied! Housing Units 

Percent of Population Living in Units with 1.01 

• Person or More Per Room . r 

Percent of Population Living in Units »cking Some 

Plumbing V W 

State Aid Ratio , 
Prior Achievement 



Process Variable^ 



J The process Variables studied came principally fEom*Vhe BH)S 
ani financia^l files, Thgy included: 



Teacher variables: ^ 
Percent Certified 
Percent with Graduate Credit 

» 

Median Cxperienoe « ' 

Perceixt Married ' , . . , » ' 
^erceat Male' * 
Median Salary 

: * • 

Financial variables: 

Total ' Expenditure 

^aenditure on Regular D^i^y Instruction 
Eiipenditure onJTeachers' ^larles 
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Expenditure on C»^htral Administration 
Expenditure- on Principals' Salaries 
Expenditure on Supervision 

District' Wealth (value of taxable property divided by 

enrol lirit nt ) 



Student variable^ : 

^ Percent At- tendance 



ProceduT^e s^ . - • 

The analyses were carried out using multiple regression models, 
following procedures fot generating and comparing ffill and/restricted 
models, l^e full model for a given criterion includes a set of non-* 
controllable variables plus the process variable of interest The _ 
restricted model is the original equation without the prof^ess variable. 
The pef-cent of variance of the criterion accounted for by the full 
model is compared- to the percent^accounted for by the restricted 
model. The diff^renqe is the portion of variance which cati be uhiqueLy 
attributed to the process variable. An F-test can be applied to test 
the significance of this unique var,iance:in th^ following manner: |^ 

F = (Ri^- R^^)/<lfi ' 



(1 - R^^)/df2 



where: = the squared multiple correlation of the full 

model (i.e., the percent of variance on the 
criterion accounted for by all the variables). 

R2^= the squared multiple correlation o^ M^S^^ i}> 
restricted model. » ^ '^l^j^U'^i^V 
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dfi= the number of linearly independent variables 
in the full model less the number in the % 
restricted model. ' 

%- < . • 

df2= the number of^ cases or observations less the 

number of .linearly independent variables in the 
full model. ' 



Results 



The teacher variables were- studied' using 736 school districts 
in New York State. This number represents about 99% of the districts 
in the state which operate schools at the third- and/or sixth-grade 
level, , * . - 

^ The teac^ variables are expi.assed either as percentages of 

the total teaching staff in the district or as medians for the 

teaching staff 'of the district. ' 

i ^ * 
Each full model was made up of a set of socioeconomic variables 

and, in some cases, prior. ^ean achievement in addition to the teacher 

variable of interest. » The restricted model was the same with the 

teachei- variable deleted. An F-test was used to test the significance 

of the difference in between the 4:w6 modelsr. Results ^e presented 

in TabU 1. Only those results for which the F-test was significant 

at the ioi level are included. This rather stringent test of 

significance was selected because the tendency of slight differences 

to be reported out of context in news media make it impo/tant not to 

report an excessl^ye number of differences which could be attributable 

to chance or to error of measurement.^' In short, an attempt was made * 

to reduce Type I errors with the resalti^; increase in Type II errors/. 
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As* can be- observed from' the table, significant results were 

obtained ov^r all of "the achievem;snt variables^ for Perc^ent Certified, 

Percent with Graduate Credit, and Median Salary. In addition, significant 

results wore obtained- in two different years for Percent with Gradu^e 

Credit. ?vrcept Certified for 1972 was not studied because the proportion 

of uncertified teachers in the state had dropped substantially,' 

drastically reducing the amount of variation observed in that variable 

in 1972. Percent Married and Percent Male produced inconsistent results 

in the two years studied. Median Experience was found to be related 

only to sixth-grade Mathematics. (DiGiovanni, March 1975, April 19,75a, 

April 1975b,)'' " 

• » , 

Expenditure variables were studied using 705 school districts. 
Financial data are collected for New York City as, a single district , 
whereas BEDS and PEP data are collected for each of the 31 co'nmiunity 
school districts. For this reason. New York City was omitted from the 
analyses casing expenditure variables. * ' 

. The analyses were carried out in much the same way as the analyses 
using teacher variables. For each expenditure variable, the full model 
(which included the expenditure Variable of interest as well as variables 
outside the control of the school district) was compared with a restricted 
model (from which the expenditure- variable was deleted). An F-test was 
used to t^st the. significance of the difference in between the full artd 
reftricted models. Results are presented in Table 2. Only those results 
for which the F-test was significant at the .01 level are included. 
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Few consisteru relationships were £ound, Total^Eac^endiCure per 
Pupil was related to Third-Grade M^th^n&tics in both 1971 and ^972. Several 
other expenditure variables were related tc Third-Grade Mathematics in 
either 1971 or'1972, ^ the relationships did hot hold up over both years* 
In ^11, expenditure variables were found to be significantly related 
to EatheiLaLics achievement in seven out o| twenty-four possible relation- 
ships. Two of the significant relationships were negative. Only one 
significant relationship was found between expenditure variables and 
reading achievement out of a possible twenty-four, (Spath, January 1975..) 

A major problem in studying expenditures is the gross nature^ of 
the variables. When data are aggrt4,cited at fhe district j^evel , specific 
effects are unlikely to be observed on a specific area of achieveaent. 
The findings might be different iT the'data Indicated how much was spent 
for a particular. set of pupils at a given grade level for a particular 
set of instructional ob5ectives. Unfortunately, these kindsof data 
were -not readily available for the' analyses de^o^bed here, 

A somewhat aore- elaborate procedure was used in studying the 
relationships of rate gf attendance to achievement. In •ddition to 
testing for linear relationships, curvilinear relationships and 
interactions between ratejifc^ttendance and several socioecononic 
variables were .ir /estigkted to detenpine their contribution to the* 
variance of ..third-grade aki sixth-grade reading and mathematics.' 
In no instance, ^ was a sign^^xnant -relationship, linear or curvllinea 
found between att^ndanc^ and Ichievement whens socioeconomic factor 
were controlJ«|. Nor were anyVf the interaction terms found to 




contribute significantly to explaining variance of any of the ac^iieve- 
ment criteria. Apparently, rate of attendance is larg^y a function 
of socioeconomic factors arid, once those factors are accounted for, * 

rate of attendance does not vatV with achievement in any systematic 

/ 

way.,. ''Spath, August 1974,)' 

A separate analysis was .carried out to study the relation of 
student mobili^vyio achievemer{% . Stepwise njultiple regression analysis 
was used rather than a model -building approach. Third-grade data, 
collected in 1972 from 35 districts, were analyzed. Results of the 
analysis failed to reveal any relationships between mobility and 
achieveoent whicli reached the .01 level of significance. (Spath, 
January 1^7 5b. ) ' - 

Conclusions ' 

Several teacher variables were found to relate- to achievement. 
fk> consistent pattern of relationships was found between expenditure 
variables and achievement. Neither rate of attendance nor student . 
nobility was found to be related to achievement when socioeconomic factors 
were accounted for. * ' . . ' 

The positive results for several teacher variables support 

logic and the conventional wisdom. The higher the salaries and the 

IDercent of certified teachers and teachers with graduate ^credit , the 

* 

higher district achievement is likely to be^ Tt^ese analyses do not 
reveal whether cause and effect are inherent in these relationships. 



14 



Nor do they indicate, if a causal relationship i_s present, the mechanism 
ich the teacher Variables influence achievement, 
Th^ of established' causality and the ^mall (though sig- 

nificant) propoi^ion of variance accounted for by the teacher variables 
stud'ied here 'suggest caution in applying the results. If school district 
policy cafls for hiring t)r rewarding teachers who are fully Certified 
and/or who he* j earned gr^u^te credit, these analyses provide sorte 
support;' but that policy, we presume, is alsjo supported by logic, ex- 
perience, and perhaps other rese^ch findings. Applying these results 
to individual cases would be especially questionable without further 
evidence. 

The analyses describeik above exemplify some .of the kinds of studj.es - 
which can be done using the di^rlct as the unit of ohse,rvation« Some 
limitations of working with district data are also evident. It was 

• \ ■• 

because of the limits of this approach that the State Education Departqient 

V 

additional studies which are being described in thi*s sypiposium, 

\ \ - 

. \ 
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